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Abstract — Recently, a unified framework for adaptive kernel based 
signal processing of complex data was presented by the authors, which, 
besides offering techniques to map the input data to complex Reproducing 
Kernel Hilbert Spaces, developed a suitable Wirtinger-like Calculus for 
general Hilbert Spaces. In this short paper, the extended Wirtinger's cal- 
culus is adopted to derive complex kernel-based widely-linear estimation 
filters. Furthermore, we illuminate several important characteristics of 
the widely linear filters. We show that, although in many cases the gains 
from adopting widely linear estimation filters, as alternatives to ordinary 
linear ones, are rudimentary, for the case of kernel based widely Unear 
filters significant performance improvements can be obtained. 



I. Introduction 

Kernel-based processing is gaining in popularity within the Signal 
Processing community [?], [?], [?], [?], [?], [?], [?], as it provides 
an efficient toolbox for treating non-linear problems. The main 
advantage of this procedure is that it transforms the original non- 
linear task, in a low dimensional space, into a linear one, that is 
performed in a higher dimensionality (possible infinite Hilbert) space 
T-L. This is equivalent with solving a nonlinear problem in the original 
space. The space H is implicitly chosen via a kernel function that 
defines the associated inner product. 

However, most of the kernel-based techniques were designed 
to process real data. Until recently, no kernel-based methodology 
for treating complex signals had been developed, in spite of their 
potential interest in a number of applications. In [?], a framework 
based on complex RKHS was presented to solve this problem. Its 
main contributions are: a) the development of a wide framework 
that allows real-valued kernel algorithms to be extended to treat 
complex data efficiently, taking advantage of a technique called 
complexification of real RKHSs, b) the elevation from obscurity of 
the pure complex kernels (such as the complex Gaussian one) as a 
tool for kernel based adaptive processing of complex signals and c) 
the extension of Wirtinger's Calculus in complex RKHSs as a means 
for an elegant and efficient computation of the gradients, which are 
involved in the derivation of adaptive learning algorithms. 

Complex-valued signals arise frequently in applications as diverse 
as communications, biomedicine, radar, etc. The complex domain 
not only provides a convenient and elegant representation for these 
signals, but also a natural way to preserve their characteristics and 
to handle transformations that need to be performed. In the more 
traditional setting, treating complex signals is often followed by 
(implicitly) assuming the circularity of the signal. Circularity is 
intimately related to the rotation in the geometric sense. A complex 
random variable Z is called circular, if for any angle both Z 
and Ze*** (i.e., the rotation of Z by angle (j)) follow the same 
probability distribution [?], [?]. Naturally, this assumption limits the 
area for applications, since many practical signals exhibit non-circular 
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characteristics. Thus, following the ideas originated by Picinbono and 
Chevalier in [?], [?], on-going research is focusing on the widely 
linear filters (or augmented filters) in the complex domain (see for 
example [?], [?], [?], [?], [?], [?], [?], [?], [?], [?], [?]). The main 
characteristic of such filters is that they exploit simultaneously both 
the original signal as well as its conjugate analogue. An important 
characteristic of widely linear estimation is that it captures the 
full second-order statistics of a given complex signal, especially in 
the case where the signal is non-circular, by considering both the 
covariance and the pseudo-covariance matrices [?], [?]. Thus, for a 
general complex signal, the optimal linear filter (i.e., widely linear) 
is linear both in z and z* . 

The main contribution of the present paper is twofold. First, we 
explore the main concepts of widely linear estimation from a new 
perspective, showing why widely linear estimation can potentially 
lead to improved performance compared to complex linear estimation. 
Moreover, we employ the framework of [?] to develop widely linear 
adaptive filters in complex RKHS, to solve nonlinear filtering tasks. 
Thus, we introduce, for the first time, the notion of widely linear 
filters in infinite dimensional Hilbert spaces. Our findings indicate 
that the "natural" choice for kernels, in the context of the widely 
linear filtering structure, are the pure complex kernels. In contrast, 
combining the widely linear structure with kernels that result from 
complexification of real kernels, does not enhance performance, 
compared to that obtained with the complexified kernel-based linear 
filterfl 

The problem of widely linear estimation using kernels has been 
also considered in [?], [?]. Although in these papers the authors 
claim that they solve the problem of augmented complex signal 
processing using kernels, a careful study reveals a lot of points that 
are not clearly described (possibly because they are addressed to a 
conference). Moreover (and this is more important) there are several 
significant inconsistencies with the traditional theory of Reproducing 
Kernel Hilbert spaces (RKHS), which is the main mathematical tool 
of any kernel based method. For example, in both papers the complex 
data are mapped to the so called feature space using a mapping 
$ : C" — >■ C*, instead of the traditional approach, which considers 
a feature map of the form $ : C" — >■ H, — k(-,z), where 

H is the RKHS induced by the kernel k. It is evident, that under 
this context, $ maps the data to an Euclidean space. Following 
this mapping of the data, the presented methods build some kernel 
matrices of the form ($(a;i), . . . , €>(cca/)). However, the case of 
treating the problem in finite dimensional Euclidean spaces, is a 
rather trivial extension of previously known results. Moreover, in 
those papers, the notion of fully complex kernels is neither mentioned 
and even more treated. However, as we prove in the present paper, 
this is the case where the augmented structure has a meaning, in 
the sense that can offer advantages. Using real kernels, under the 
complexification theory, in the context of augmented filters, offers 
no advantages. 

The paper is organized as follows. We start with a brief introduction 
to complex RKHSs and Wirtinger's Calculus in Section|ll] In Section 
Unl we describe the concept of widely linear estimation and show why 

'This is because complexification implicitly adds a conjugate component 
to the adopted model. 
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this is better thian complex linear estimation. Finally, widely linear 
kernel based adaptive filters are described in section [W] Experiments 
are provided in section |V] Section [Vll concludes the paper. 

II. Preliminaries 

Throughout the paper, we will denote the set of all integers, real 
and complex numbers by N, R and C respectively. Vector or matrix 
valued quantities appear in boldfaced symbols. A RKHS [?] is a 
Hilbert space H over a field F for which there exists a positive definite 
function k : X x X — > F with the following two important properties: 
a) For every x £ X, x) belongs to H and b) k has the so called 
reproducing property, i.e., f{x) = {f,K{-,x))n, for all / G H, in 
particular K{x,y) — {K.{-,y), k{-, x)}^- The map $ : X — >■ H : 
— k{-,x) is called the feature map of H. 

Although the underlying theory has been developed by the math- 
ematicians for general complex reproducing kernels and their as- 
sociated RKHSs, mostly the real kernels have been considered by 
the machine learning and signal processing communities [?], [?]. 
Some of the most widely used kernel in the literature are the 
Gaussian RBF, i.e., n^^-g,d {x, y) :— exp ^— '^i'=i(^k-yk) defined 
for x,y G R'^, where a is a free positive parameter and the 
polynomial kernel: Kd{x,y) :— (l + a3"^j/) , for d G N. Many 
more can be found in [?], [?]. Complex reproducing kernels, that 
have been extensively studied by the mathematicians, are, among 
others, the Szego kernels and the Bergman kernels. Another important 
complex kernel is the complex Gaussian kernel, which is defined 
as: n^ f,4z,w) := exp Sfc=i(^fc-"'fc)'' ^ ^ where z,w € C^, Zk 

denotes the fc-th component of the complex vector z G and exp(-) 
is the extended exponential function in the complex domain. 

To generate kernel adaptive filtering algorithms on complex do- 
mains, according to [?], one can adopt two methodologies. A first 
straightforward approach is to use directly a complex RKHS, using 
one of the complex kernels given in section |II| and map the original 
data to the complex RKHS through the associated feature map 
= k{-,x). Another alternative technique is to use real kernels 
through a rationale that is called complexification of real RKHSs. 
This method has the advantage of allowing modeling in complex 
RKHSs using popular well-established and well understood, from a 
performance point of view, real kernels (e.g., gaussian, polynomial, 
e.t.c). In the first case, we map the data directly to the complex 
RKHS, using the corresponding complex feature map $(z) = 
Kc{-,z), while in the complexification scenario we employ the map 
#(z) = ^(x, y)+i(t(x, y), where z = x + iy, and $ is the feature 
map of the chosen real kernel kr, i.e., ^{x, y) = {x, y)). 

In order to compute the gradients of real valued cost functions, that 
are defined on complex domains, we adopt the rationale of Wirtinger's 
calculus [?]. This was brought into light recently [?], [?], [?], [?], 
[?], as a means to compute, in an efficient and elegant way, gradients 
of real valued cost functions that are defined on complex domains 
(C). It is based on simple rules and principles, which bear a great 
resemblance to the rules of the standard complex derivative, and it 
greatly simplifies the calculations of the respective derivatives. The 
difficulty with real valued cost functions is that they do not obey the 
Cauchy-Riemann conditions and are not differentiable in the complex 
domain. The alternative to Wirtinger's calculus would be to consider 
the complex variables as pairs of two real ones and employ the 
common real partial derivatives. However, this approach, usually, is 
more time consuming and leads to more cumbersome expressions. 
In [?], the notion of Wirtinger's calculus was extended to general 
complex Hilbert spaces, providing the tool to compute the gradients 
that are needed to develop kernel-based algorithms for treating 
complex data. In the same paper, the aforementioned toolbox was 



employed in the context of the complex LMS and two realizations 
of the complex kernel LMS algorithm were developed. The first 
one, which is denoted as NCKLMSl adopts the complexification 
procedure and the second one, which is denoted as NCKLMS2, uses 
the complex gaussian keme0. 

III. Widely linear estimation filters 

As mentioned in the introduction, ongoing research in complex 
signal processing is mainly focused on the so called widely linear 
estimation filters, or augmented filters, as they are also known. These 
are filters that take into account both the original values of the signal 
data and their conjugates. For example, in a typical LMS task, we 
estimate the output as d{n) — z and the step update as w{n) = 
win — 1) + pe* {n)z{n). In this case, d{n) is a linear estimation 
filter. However, the linearity property is taken with respect to the 
field of complex numbers. Picinbono and Chevalier, in [?], proposed 
an alternative approach. They estimated the filter's output as d{n) = 
z+v^ z* and showed that it provides better results in terms of the 
mean square error. This, of course, is expected since d{n) provides 
a more rich representation than d(n). On the other hand, d{n) is 
no longer linear over the field C. It is linear, however, over the real 
numbers R. To emphasize this difference, in the relative literature, 
d(n) is often called C-linear, while d{n) is called R — linear. 

In [?], [?], [?], it is shown that the widely linear estimation filter 
is able to capture the second order statistical characteristics of the 
signal which are essential, for non-circular sources. Although in many 
relative works this is highlighted as the main reason for adopting 
widely linear techniques, in this paper, we will highlight a different 
perspective. 

Our starting point will be the definition of linearity. Prior to it, it 
should be clarified, that complex processing is equivalent with pro- 
cessing two real signals in the respective Euclidean (Hilbert) spaces. 
The advantage of using complex algebra is that the algorithm and/or 
the solution may be described in a more compact form. Moreover, 
the complex algebra allows for a more intuitive understanding of the 
problem, as many geometric transformations can be easily described 
using complex algebra in an elegant way. Finally, the application of 
Wirtinger's calculus greatly simplifies the calculations needed for the 
gradients of real valued cost functions. 

Having this in mind, we now turn our attention to a typical complex 
LMS task. Let z{n) G C and d{n) G C be the input and the output 
of the original filter. We estimate the output of the filter using a C- 
linear response d{n) = z(n). The typical complex LMS task 
aims to compute w G C, such that the error E\\d{n) — d(n)p] is 
minimized. If we set w = lu,- + iwi and z = x -\- iy, we take that 

d{n) = w"^ X + wf y + i(w'^ y — wf x) . (1) 

However, the real essence behind a complex filter operation is the 
following: Given two real vectors, x{n) and y{n), compute linear 
filters in order to estimate two real processes, dr{n) and di{n), in 
an optimal way, that jointly cares for both dr(n) and di{n). Let 
us express the problem in its multichannel formulation, using real 
variables only, i.e., 

/J(n)\ /nf,, ul,\ (x\^^ {x\ 
\d,(n)J \uii uiij \yj \yj 

From a mathematical point of view, this is the definition of a linear 
operator from R'^" — ^ R'^. The elements of U are computed such that 
both \dr{n) — dr{n)\'^ and \di{n) — di{n)\^ are jointly minimized. 
This leads to the so called Dual Real Channel (DRC) formulation. 

^Due to the fact that there are a lot of algorithms mentioned in the paper, 
Table |l] presents their acronyms for the reader's conveniance. 
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Thus, we take the relations dr{n) = uf^icc + Ui 2y and di{n) = 
U2_iX + u^^2V- In complex notation, if we consider that d{n) = 
dr{n) + idi(n), we obtain the expression 



with covariance matrix V — 



d{n) 



T 

Ui iX 



(2) 



It is easy to see that the DRC approach expressed by relation (|2} 
adopts a richer representation than that of the traditional LMS in (TJ. 
Moreover, it takes a few lines of elementary algebra to show that an 
equivalent expression of (O is the widely linear estimation filter, i.e., 
d{n) — d{n). This will be our kick off point to define the task in a 
general Hilbert space. In general, we can prove the following: 

Proposition 1. Consider a rea0 Hilbert space T-L, the real Hilbert 
space H and the complex Hilbert space H = H + iH. Then any 
continuous linear function T : T-L^ — >■ can be expressed in 
complex notation as 

T{x, v) = T{x + iy) = T{z) ^ {z,w)^ + {z' , v),_^ , (3) 

for some w,v G H, where {■,-)m is the respective inner product of 
H. 

Proof: Recall that any element of or can be ex- 
pressed either as a vector or as a complex element (complex 
notation). Any function T : — > takes the form 

T{x,y) = {Ti{x,y),T2{x,y))'^. As T is linear, both Ti, T2 
are also linear. Thus, as a consequence of the Riesz's represen- 
tation theorem, there is ui — (ufi,u^2)"^ £ such that 
Ti{x,y) = {(a;,y)^, (uJ^i,uf^2)^)-H2- Similarly, T2{x,y) = 
((a;, y)"^, (u^i,U2,2)^)7i2, for some U2 = 112,2)^ G 
Thus, in complex notation, T can be written as 

T{x,y) = Ti{x,y)+iT2{x,y) 

= {x,ui,i)n + {y,ui,2)n +i{{x,U2.i)n + {y,U2,2)n)- 

(4) 



We define wi,'W2,v-i_,V2 



H as follows: wi — 



"l,l+"2.2 



102 



"1,2— "2,1 



Vl 



2 ' ^ — 2 

"2 1+"1 2 T^u I 

- • „ ' . Then, ui,i = wi + vi, ui,2 



V2 



W2 



V2, 



1*2,2 = wi — Vl and U2,i — —W2 — V2- Substituting in l|4j 
we take: 

T{z) =T{x, y) = {x, wi)n + {x, vi)n + {y, W2)h - {y, V2)n 
+ i (-{x,w2}n - {x,v2}n + {y,wi}n - {y,vi)n) 

= {x + iy, wi + iw2)m + {x - iy, vi + iv2)m 

= (z,io)h + {z* ,v)'a, 
where z = x -\- iy, w = wi + iw2, v = vi + iv2- □ 

Remark 1. In view of proposition [7] one understands that the 
original formulation of the complex LMS was rather "unorthodox" , 
as it excludes a large class of linear functions from being considered 
in the estimation process. It is evident, that the linearity with respect 
to the field of complex numbers is restricted, compared to the linearity 
that underlies the DRC approach, which is more natural. Thus, the 
correct complex linear estimation is T{z) — {z,w)^:^ + 
rather than T{z) = {z, w}^. 

Remark 2. Any ^-linear function can be expressed as in 0. 

Remark 3. Let e{n) — d{n) — d{n), be the error of the widely linear 
estimation, where d{n) is the desired response. If we assume that the 
error follows the complex normal distribution of van den Boss [?], 

^By the term real (complex) Hilbert space, we mean a Hilbert space over 
the field of real numbers K (complex numbers C). 



1 




, we can easily derive that the 



maximum likelihood estimator, for w,v is equivalent with minimizing 
the square error, as it is the case in widely linear LMS. 



IV. Widely Linear estimation in complex RKHS 

In this section, we will develop realizations of the Augmented Com- 
plex Kernel LMS (ACKLMS) algorithm using either pure complex 
kernels, or real kernels under the complexification trick. We show 
that ACKLMS offers substantial improvements versus complex kernel 
LMS (CKLMS), when the complex gaussian kernel is employed. 
On the other hand, the ACKLMS, which is developed under the 
complexification trick, degenerates to the standard CKLMS. 

Consider the sequence of examples d{l)), {z{2),d{2)), . . . , 

(^(iV), d{N)), where d{n) G C, z{n) G C, z{n) = x{n) + iy(n), 
x{n), y{n) G R", for n = 1, . . . , Af. Consider, also, a real RKHS H, 
the real Hilbert space and the complex Hilbert space H — T-L+iH. 
We model the widely linear estimation filter in H as: 

d{n) = {^{z{n)),w{n - 1))^ + {z{n)),v{n - , (5) 

where $ is an appropriate function that maps the input data to the 
feature space H. This is equivalent with transforming the data to a 
complex RKHS and applying a widely linear complex LMS to the 
transformed data. The objective of the ACKLMS is to estimate, w 
and V, so that to minimize E [£,„(w)], where Cntw) = je(n)p — 
I ~ !^ 

d(n) — d(n) , at each time instance n. 
Applying the rules of Wirtinger's calculus in complex RKHS, 



we can easily deduce that = —^{z{n)) ■ e*(r 

-**(z(n)) • e*(n). Thus, the step updates of the ACKLMS are 
w{n) = w{n — 1) + jj,^{z{n))e* (n), v{n) — v{n — 1) + 
/i^*(z(n))e*(n). Assuming that t<;(0) = v(0) = 0, the repeated 
application of the weight-update equations gives: 



n-1 

v{n - 1) = **(z(fc))e*(fc). 



(6) 
(7) 



Combining ^ and (|7) leads us to conclude that the filter's output, 
at iteration n, becomes: 

n-l 

d{n) =M E *(^(fc)))B 

1 



Recall that in the complexification trick the associated function 
that maps the data to H is given by ^(z) — ^{x, y) + i$(a;, y) — 
Km{-, {x, y)) + iKR{-, {x, y)), where $ is the feature map of T-L and 
kr its respective real kernel. Under this condition, the filter output 
at iteration n takes the form 

n-l 

d{n) = 4n^e{k) ■ Km{{x{k),y{k)),{x{n),y{n))) . (8) 

fc=i 

This is exactly the same formula, which was obtained in the case of 
the complexified complex kernel LMS (except a rescaling). Thus, we 
deduce that, in this case, the standard complexified NCKLMSl pre- 
sented in [?] and the complexified augmented CKLMS are identical. 
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Name 


Description 


NCLMS 


Normalized Complex LMS 


NACLMS 


Normalized Augmented Complex LMS 
(Widely-Lineal') 


NCKLMS 1 


Normalized Complex Kernel LMS 
(complexified real kernels) 


NCKLMS2 


Normalized Complex Kernel LMS 
(pure complex kernels) 


NACKLMS 


Normalized Augmented Complex Kernel LMS 
(pure complex kernels) 


CNGD 


Complex Non-linear Gradient Descent [?] 


MLP 


Multi Layer Perceptron 
(50 nodes in the hidden layer) 



TABLE I 

Acronyms of the Algorithms used in the paper. 



On the other hand, for the case of a pure complex kernel kc, the 
fiher output becomes 

n— 1 n—l 

d{n) = ^l^e{k)Kc{z{k),z{n)) + e(k)Kc{z{k), z{n)), 
fc=i fc=i 

(9) 

as ($(z), $(c))jj — Hic{z,c), for any z,c G C^. In this case, it 
is evident that the augmented CKLMS (ACKLMS) will result to a 
different solution compared to that of NCKLMS2, as it exploits a 
richer representation. From (|9), we deduce that the complexity of 
the normalised ACKLMS is of the same order as the complexity of 
NCKLMS2 and NCKLMS 1. 

V. Experiments 

The performance of the normalized augmented CKLMS 
(NACKLMS) has been tested in the context of a nonlinear channel 
equalization task. As in [?], two nonlinear channels have been 
considered. The first channel (labeled as soft nonlinear channel in 
the figures) consists of a linear filter: t[n) = (—0.9 + 0.8i) • 
s(n) + (0.6 — 0.7i) • s(n — 1) and a memoryless nonlinearity 
q{n) = t{n) + (0.1 + 0.15i) ■ t^{n) + (0.06 + 0.05i) • r\n). 
The second one (labeled as strong nonlinear channel in the figures) 
comprises of the consists of the linear filter: t{n) = (—0.9 + 0.8i) • 
s(n) + (0.6- 0.7i) -5(71-1) + (-0.4+0.3i) -8(71- 2) + (0.3- 0.2i)- 
s(n — 3) + (— O.li — 0.2i) • s{n — 4)) and the nonlinearity: q{n) = 
tin) + (0.2 + 0.25i) • t^{n) + (0.08 + 0.09i) ■ t^{n). At the receiver 
end of the channels, the signal is corrupted by white Gaussian noise 
and then observed as r(n). The level of the noise was set to 15dB. 
The input signal that was fed to the channels had the form s(n) = 
0.70 ^\/l — p2X(n) + ipF(n)^, where X{n) and Y(n) are gaus- 

sian random variables. This input is circular for p = \/2/2 and highly 
non-circular if p approaches or 1 [?]. To solve the channel equal- 
ization task, we apply the NACKLMS algorithm to the set of samples 
((r(n + D),r{n + D -l),...,r{n + D - L + l)), s{n)), where 
1/ > is the filter length and D the equalization time delay. 

Experiments were conducted on 100 sets of 5000 samples of 
the input signal considering both the circular and the non-circular 
cases. The results are compared with the NCLMS and the NACLMS 
(i.e.. Normalized Augmented CLMS, or widely linear NCLMS as 
it is sometimes called) algorithms as well as two adaptive nonlinear 
algorithms: a) the CNGD algorithm [?] and a Multi Layer Perceptron 
(MLP) with 50 nodes in the hidden layer [?]. In both cases, the 
complex tanh activation function was employed. Figure [T] shows 
the learning curves of the pure complex kernel LMS (labeled as 
NCKLMS2 in the figures) [?] and the proposed NACKLMS, using 
the complex Gaussian kernel (with a = 10), together with those 



obtained from the NCLMS and the WL-NCLMS algorithms. Figure 
|2] shows the learning curves of NCKLMS2 and NACKLMS, using 
the complex Gaussian kernel (with a — 15), versus the CNGD and 
the L-50-1 MLP for the hard non-linear channel. 

The novelty criterion (see [?], [?]) was used for the sparsification 
of the NCKLMS2 and NACKLMS with (5i = 0.1 and S2 = 0.2. 
In both examples, NACKLMS considerably outperforms the linear, 
widely linear (i.e., NCLMS and WL-NCLMS) and nonlinear (CNGD 
and MLP) algorithms (see figures [T]|2ll. The NACKLMS also exhibits 
improved performance compared to the NCKLMS2 for non-circular 
input sources. Moreover, observe that while the gain of the NACLMS 
against NCLMS is rather rudimentary (smaller than 0.2 dB), the gains 
of NACKLMS over NCKLMS2 are significant (approximately 2dB). 
For circular signals, the two models (NCKLMS2 and NACKLMS) 
lead to almost identical results, as expected [?], [?]. 

VI. Conclusions 

In this paper we proposed a method for widely linear estimation 
in complex RKHS, based on the framework presented in [?]. The 
developed rationale was applied to derive the augmented complex 
kernel LMS (NACKLMS). Moreover, some important properties of 
widely linear estimation were discussed to provide a further under- 
standing of the underlying toolbox. Experiments of the developed 
NACKLMS, for both circular and non-circular input data, showed a 
significant decrease in the steady state mean square error, compared 
with other known linear, widely linear and nonlinear techniques. 
Both theoretical and experimental results showed the importance 
of employing augmented filters versus linear ones in the complex 
domain. 
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Fig. 2. Learning curves for NCKLMS2 = 1/8), NACKLMS, (/i = 1/8, cr = 15), MLP (fj. = 0.0003) and CNGD = 0.0005) (filter lengtli L = 5, 
delay D = 2) for the hard nonlinear channel equalization problem, for (a) the circular input case, (b) the non-circular input case (p = 0.1). 



